Convergence of Backpropagation with Momentum for Network Architectures with Skip Connections
نویسندگان
چکیده
منابع مشابه
Convergence of Split-Complex Backpropagation Algorithm with Momentum∗
This paper investigates a split-complex backpropagation algorithm with momentum (SCBPM) for complex-valued neural networks. Some convergence results for SCBPM are proved under relaxed conditions compared with existing results. The monotonicity of the error function during the training iteration process is also guaranteed. Two numerical examples are given to support the theoretical findings.
متن کاملLearning Neural Network Architectures using Backpropagation
Deep neural networks with millions of parameters are at the heart of many state of the art machine learning models today. However, recent works have shown that models with much smaller number of parameters can also perform just as well. In this work, we introduce the problem of architecture-learning, i.e; learning the architecture of a neural network along with weights. We start with a large ne...
متن کاملSkip Connections Eliminate Singularities
Skip connections made the training of very deep networks possible and have become an indispensable component in a variety of neural architectures. A completely satisfactory explanation for their success remains elusive. Here, we present a novel explanation for the benefits of skip connections in training very deep networks. The difficulty of training deep networks is partly due to the singulari...
متن کاملSkip Connections Eliminate Singularities
Skip connections made the training of very deep networks possible and have become an indispensable component in a variety of neural architectures. A completely satisfactory explanation for their success remains elusive. Here, we present a novel explanation for the benefits of skip connections in training very deep networks. The difficulty of training deep networks is partly due to the singulari...
متن کاملInvestigation of Mechanical Properties of Self Compacting Polymeric Concrete with Backpropagation Network
Acrylic polymer that is highly stable against chemicals and is a good choice when concrete is subject to chemical attack. In this study, self-compacting concrete (SCC) made using acrylic polymer, nanosilica and microsilica has been investigated. The results of experimental testing showed that the addition of microsilica and acrylic polymer decreased the tensile, compressive and bending strength...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of Computational Mathematics
سال: 2021
ISSN: ['2456-8686']
DOI: https://doi.org/10.4208/jcm.1912-m2018-0279